Jasmine Adams
December 12, 2022
Your weather app doesn’t always match what you see when you look out the window. Despite considerable advances in weather reporting, the National Weather Service (NWS) — the primary data source for all weather service providers in the United States — struggles to maintain comparable accuracy and precision during periods of high volatility. Moreover, satellites, radars, and other forecasting tools synthesize complex weather patterns across large geographic areas and are not well suited for observing sudden changes in surface-level precipitation. In fact, radar – the NWS’s primary tool for measuring precipitation observed on the ground – does not measure surface-level rainfall at all. Meteorologists infer how much it’s raining on the ground by emitting electromagnetic energy and analyzing the “echo” that precipitation particles reflect back.
United States National Radar
While this approach can provide valuable information about overall
precipitation patterns, detecting sudden shifts in local weather events
requires eyes on the ground. As such, the NWS must rely on citizen
volunteers to understand timely changes in severe weather condition and
daily precipitation observed at the surface.
Using radar
to estimate observed weather conditions isn’t exactly a Watergate-level
scandal (especially considering I learned this information from NWS
website). However, whether the NWS can provide timely, local weather
warnings during periods of increased weather volatility hinges on its
ability to accurately assess conditions on the ground. This mandate has
incredible implications here in Washington, D.C., where urban topography
and impervious surfaces, along with rising temperatures induced by
climate change, have increased rainfall volatility and interior flooding
(Cone, 2012; DC Silver Jackets, 2017; Zahura & Goodall, 2022).
Using my own personal weather station (PWS) in Dupont
Circle (Figure 1), I compare NWS hourly rainfall estimates to observed
rainfall at home to assess whether NWS estimates for the greater D.C.
area representative of precipitation observed at the neighborhood
level.
Figure 1 - Ambient Weather WS-7078 Smart Weather Station
| Sensory Array |
|---|
| 1 Antenna |
| 2 Rain collector |
| 3 UV / light sensor |
| 4 Mounting pole |
| 5 Mounting Base |
| 6 Balance indicator |
| 7 Wind cups |
| 8 Radiation shield |
| 9 Wind vane |
Weather Underground reports live and historical weather data from a
worldwide public network of personal weather stations. To keep track of
my station’s reported weather conditions, I registered it on wundergound.com on November 13
and began tracking outdoor weather conditions the following day.
Official
data for the Washington, D.C. Metropolitan area are estimated via
radar at the Washington/Baltimore regional weather station located at
Reagan Nation Airport. Data are reported hourly and are not available
beyond three days through any public source. That meant I had to grab
data from their website at least once every three days to avoid any
critical gaps in information. While I managed to accomplish this feat,
the same cannot be said for my weather station at home.
Home rainfall data for November 27 are missing due to human error (i.e.,
my roommate mistakenly moving the station just under our awning per my
admittedly ambiguous instructions). To remedy this shortcoming and
mitigate other potential measurement errors of which I am not aware, I
cross reference my home data with data from two other nearby personal
weather stations in the Weather Underground network. The stations are
located in Adams Morgan and just east of Dupont Circle.
Unsurprisingly, this glorified science fair experiment came with some challenges and limitations. As alluded to before, radars and rain gauges measure rainfall through methods that are not apples to apples. While my home station and the Adams Morgan station have a rain gauge accuracy of ±7% and ±10% respectively, accuracy for the station east of Dupont and the radar at Reagan Nation Airport are unknown. Moreover, with limited information and quality control checks on on other personal weather stations set ups, its plausible that they are also vulnerable to a host of other third party factors compromising the accuracy of their reports.A summary of data and design limitations are thus presented in Table 1.
| Weather Station | NWS | Home | PWS1 | PWS2 |
|---|---|---|---|---|
| Station Location | Regean Airport | N of Dupont | E of Dupont | Adams Morgan |
| Station Type | Unknown | WS-7078 | Unknown | WS-1400-IP |
| Method of Rain Detection | Radar | Rain Gauge | Rain Gauge | Rain Gauge |
| Position of Detected Rainfall | Closer to Clouds | Surface Level | Surface Level | Surface Level |
| Measurement Accuracy | Unknown | ± 7% | Unknown | ± 10% |
| Risk of Roommate/Squirrel Interference | Low | High | Medium | Medium |
I copied data from tables on the NWS and Weather Underground websites
from November 14 - December 10 and placed them into excel for
preliminary data cleaning. I then uploaded data into R where I filtered
for hours in which at least one station recorded 0.01 inches of rain or
more. After gathering descriptive statistics for hours in which it
rained (Table 2), I compare recorded precipitation amounts between each
station using a series of OLS linear regressions. I also ran regressions
to assess whether variation in recorded temperature, pressure, and
humidity is comperable to variation in measured precipitation.
Although variation in measured rainfall between each station may be minimal, there is a statistically significant difference in rainfall reported by the NWS station and each of the other local stations.
Differences in measured rainfall between the three neighboring stations are smaller than between these stations and the NWS station.
Variation between my home station and the NWS station is greater for reported rainfall than for any other metric.
There have been approximately seven rainy days since data collection began on November 14. Across those 7 days, there are 45 hours for which at least 0.01 inches of rain is recorded (that is 39 hours of recorded rainfall at home given data for November 27 are missing. Although this sample size is large enough for a statistical analysis, it is still quite small. Daily accumulated precipitation and the average among all stations are displayed in Table 2.
| Date | NWS (DCA) | Home (Dupont) | East Dupont | Adams Morgan | Avg. Accum. |
|---|---|---|---|---|---|
| Nov 30 | 0.32 | 0.37 | 0.40 | 0.35 | 0.360 |
| Nov 27 | 0.24 | NA | 0.23 | 0.20 | 0.222 |
| Nov 25 | 0.15 | 0.17 | 0.17 | 0.19 | 0.170 |
| Nov 15 | 1.22 | 2.56 | 0.85 | 1.39 | 1.502 |
| Dec 7 | 0.05 | 0.01 | 0.02 | 0.02 | 0.025 |
| Dec 6 | 0.17 | 0.15 | 0.21 | 0.19 | 0.179 |
| Dec 3 | 0.31 | 0.31 | 0.37 | 0.32 | 0.326 |
Graphing measured precipitation in Figure 2 illustrates how, aside from
data collected on November 15, measured rainfall between local stations
hovers around the same amount. Granted, the visualization suggests that
whether I keep or disregard outliers, particularly the two heighest
values measured by the home station, may notably change the estimated
difference in hourly rainfall between stations.
A quick boxplot reveals that there are five outliers ranging from approx. 0.22 - 0.8 that I would be statistically justified in removing. However, due to the small sample size, I elect to remove only the two data points above 0.6 inches.
(Home Weather Station)
Like Figure 2, the line graph in Figure 4 represents stations’
measured hourly rainfall, excluding outlying values of 0.69 or
above.
| NWS Average | Home Average | p-value |
|---|---|---|
| 0.057 | 0.092 | 0.253 |
| NWS Average | EDup Average | p-value |
|---|---|---|
| 0.055 | 0.05 | 0.707 |
| NWS Average | AdMo Average | p-value |
|---|---|---|
| 0.055 | 0.059 | 0.769 |
| NWS | NWS | NWS | Home | Home | Adams Morgan | |
|---|---|---|---|---|---|---|
| Home | 0.06 *** | |||||
| (0.01) | ||||||
| E. Dupont | 0.06 *** | 0.14 *** | 0.07 *** | |||
| (0.00) | (0.02) | (0.01) | ||||
| Adams Morgan | 0.06 *** | 0.16 *** | ||||
| (0.00) | (0.01) | |||||
| N | 39 | 45 | 45 | 39 | 39 | 45 |
| R2 | 0.76 | 0.79 | 0.93 | 0.66 | 0.85 | 0.82 |
| All continuous predictors are mean-centered and scaled by 1 standard deviation. *** p < 0.001; ** p < 0.01; * p < 0.05. | ||||||
Finally, when comparing all aspects of the weather measured by both
statiions, it appears that the temperature, dew point, pressure, and
humidity measured by both stations is highly comparable, with data
measured by one station accounting for 91% - 100% of the variation in
data collected by the other. Given that rainfall measurd by the NWS
station accounts for only 76% of the rainfall measured by the Home
station, the final hypothesis is correct.
| NWS Rain | NWS Temp | NWS Dew | NWS Pressure | NWS Humidiity | |
|---|---|---|---|---|---|
| Home Rain | 0.06 *** | ||||
| (0.01) | |||||
| Home Temp | 5.31 *** | ||||
| (0.18) | |||||
| Home Dew | 4.88 *** | ||||
| (0.23) | |||||
| Home Pressure | 0.14 *** | ||||
| (0.00) | |||||
| Home Humidity | 7.71 *** | ||||
| (0.41) | |||||
| N | 39 | 39 | 39 | 39 | 39 |
| R2 | 0.76 | 0.96 | 0.92 | 1.00 | 0.91 |
| All continuous predictors are mean-centered and scaled by 1 standard deviation. *** p < 0.001; ** p < 0.01; * p < 0.05. | |||||
Results suggest that measured rainfall is not distinctly different
between the NWS station at Reagan National Airport and the other
personal stations in the Dupont Circle and Adams Morgan area. Granted,
coming to that conclusion using these results alone may be jumping the
gun, as there are many limitations to this study. Not only are the
weather stations’ sample sizes quite small and measurement errors
unknown, but I conduct this study in the Fall when temperatures are
cooler and rainfall is generally less volatile.
I recommend
that future studies use year round data, or at least data collected
during warmer periods, as that could reveal a different picture about
how well NWS weather station reports reflect weather reported on the
surface. Still, it’s promising that these initial results suggest that
NWS radar predictions may not too far off.